Autonomous transportation system and methods

ABSTRACT

Autonomous and manually operated vehicles are integrated into a cohesive, interactive environment, with communications to each other and to their surroundings, to improve traffic flow while reducing accidents and other incidents. All vehicles send/receive messages to/from each other, and from infrastructure devices, enabling the vehicles to determine their status, traffic conditions and infrastructure. The vehicles store and operate in accordance with a common set of rules based upon the messages received and other inputs from sensors, databases, and so forth, to avoid obstacles and collisions based upon current and, in some cases, future or predicted behavior. Shared vehicle control interfaces enable the AVs to conform to driving activities that are legal, safe, and allowable on roadways. Such activities enable each AV to drive within safety margins, speed limits, on allowed or legal driving lanes and through allowed turns, intersections, mergers, lane changes, stops/starts, and so forth.

REFERENCE TO RELATED APPLICATIONS

This invention application claims priority to, and the benefit of, U.S.Provisional Patent Application Ser. No. 62/701,152, filed Jul. 20, 2018,the entire content of which is incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates generally to autonomous vehicles (AVs) and, inparticular, to autonomous driving systems (ADS) that support autonomoustransportation with digital highways and infrastructure to achieveultimate safety goals and fail-safe operation.

BACKGROUND OF THE INVENTION

Autonomous driving is actually not that new. The enabling technology forsophisticated, human-capable vehicle robotics has been developed overthe last 30 years, beginning with the pioneering work done in theAutonomous Land Vehicle (ALV) program funded by DARPA starting in theearly 1980s. The earliest work was done at Carnegie-Mellon University,the Environmental Research Institute of Michigan, MIT, Stanford ResearchInstitute, and others, integrated into the DARPA Autonomous Land Vehicleprototype by Martin Marietta in Denver.

Lowrie [1985] provides a good overview of the ALV and describes avehicle control substantially the same as disclosure Smid et al. U.S.Patent Application 20080262669 [2008]. Kanade, Takeo, Thorpe, andWhittaker [1986] describe CMU contributions to ALV, which include adescription of using 3D LADAR developed by ERIM as instrumental inobstacle detection and avoidance. Carmer, et al. [1996] reviews thistechnology from the perspective of 10 years of further work andunderstanding. Shafer, Stentz, and Thorpe [1986], Nasr, Bhanu, andSchaffer [1987]. Turk et al, [1987], Waxman et al. [1987], Asada [1988],Daily [1988], Dunlay [1988], and Turk et al. [1988] describe ALV andNavlab (a robotic vehicle developed in parallel to ALV as aCarnegie-Mellon testbed) basic architecture and parts of an autonomousdriving vehicle, including roadway identification and following (lanekeeping), path planning and re-planning, obstacle detection andavoidance (heavily dependent upon 3D laser radar data or 3D computervision to capture the three dimensional nature of obstacles), automotivecontrols (electric steer, speed control, etc.) and vehicle location(inertial and later GPS-based location estimation). This work disclosesessentially all the features and architecture of subsequent work upthrough that reported in the DARPA Urban Challenge (2007 and 2008) andwork presently being conducted by Google.

The Wikipedia article “Autonomous Car” (per Dec. 23, 2012 seehttp://en.wikipedia.org/wiki/Autonomous_car and DARPA maintained UrbanChallenge archive at http://archive.darpa.mil/grandchallenge/) providesa good alternative overview of this prior art. This technologydemonstrated inertial localization fused with GPS for self-location ofvehicles, path planning from a map database, path plan to path executionthrough a vehicle control interface, and various vehicle-to-vehicle, andvehicle-to-infrastructure behaviors (passing, obstacle avoidance, speedlimits, lane keeping, lane change prohibition segments, right-of-way,pass-through and n-way stop intersections, traffic circles, open areanavigation, parking, speed pacing/spacing maintenance to a lead vehicle,u-turns, re-planning when routes are determined to be blocked). Thesalient aspect of this prior work is that it assumed good weatherconditions (typically indoor testing and outdoor testing ordemonstrations in dry clear weather, sometimes in the day and night, butnot in rain, snow, or flooded conditions).

Later programs that followed ALV include Army Demo I, DARPA Demo II, andagain Army Demo III. While the automated driving built under theseprograms improved, this improvement was less conceptual innovation andmore that the base technology in 3 dimensional imaging, GPS, computingpower, and open terrain path planning substantially improved so earlierarchitecture could be more successfully realized. These programs arereview by Shoemaker [2006], the government's program manager for Demo I,part of Demo II, and initially Demo III—also by and Matsumura et al.[2000]. Demo II (circa 1996) and III (circa 2001) included all aspectsof a modern self-driving vehicle, including GPS/Inertial navigation, 3Dcomputer vision and ladar obstacle detection and avoidance, routeplanning based on mission maps, WiFi-like radio network operatedtelemetry, and vehicle controls by wire (electronic speed, steering andother vehicle function controls).

Everett et al. [1992], Chun et al. [1995], and Carmer et al. [1996]specifically describe aspects of how path planning, obstacle detection,and collision avoidance were done in these programs. In parallel to thework done in the US, German and EU funded work beginning with thepioneering high speed roadway driving demonstrations by Dickmanns et.al. [1985] and later in the 1980s show this technology in the civiliansetting on German roadways (in contrast to the ALV and Demos I-III workin the US which focused on off-road and over less developed roads). TheArmy and DARPA Demos I, II, and III focused on off-road driving, butalso in dry and favorable weather conditions. The European work focusedmore on road and lane driving, but also during good weather conditions.

Perhaps the capstone in the effort to develop automated vehicles was thewell publicized DARPA Grand Challenges, culminating in 2007 with theUrban Challenge [DARPA 2007]. In this highly publicized challenge, 35semifinalist were test in Victorville, Calif. in November of 2007. Overtwenty substantially met DARPA's automated driving criteria whichincluded California driving rules [DARPA Rules Section 5.5.1], speedlimit controlled driving [DARPA Rules Section 1.7 “adherence to speedlimits”], proper lane position keeping [DARPA Rules Section 1.7 “Followpaved and unpaved roads and stay in lane”], proper position relative toother vehicles on the road [DARPA Rules Section 1.7 “Change lanes safelywhen legal and appropriate, such as when passing a vehicle or enteringan opposing traffic lane to pass a stopped vehicle. Vehicles must notpass other vehicles queued at an intersection” and DARPA Rules 5.2.3,B.4 “maintaining a minimum spacing equal to the forward vehicleseparation distance”], proper right of way behavior at intersections[DARPA Rules 5.2.3 B2. “Intersection precedence,” 5.2.5 D.2 “Merge,” andD.4 “Left turn”], speed merge and passing [DARPA Rules 5.2.2 A.10“Leaving lane to pass” and A11. “Returning to lane after passing”],general route planning to a mission specification [DARPA Rules 5.2.4 C4“Dynamic Planning”], route execution with dynamic obstacle detection andavoidance [DARPA Rules C.2 “demonstrates ability to negotiate obstaclefield safely”], and operating under conditions of variable ornon-performance of GPS [DARPA Rules 5.2.4 C.6 “GPS Outage”].

Demonstrations in California in 2007 were performed by over 20 teams,but were also in dry dessert weather conditions (no snow, running water,ice, or fog/dust). This was a milestone showing that practicalapplications of human-safe automated driving were possible at thecurrent state of the art (circa 2007). Later disclosures describeversions of the basic AV architecture, controls, and lane/path planningand obstacle detection behaviors originally disclosed in the suite ofpapers published by teams that participated in the DARPA Urban Challenge(per http://archive.darpa.mil/grandchallenge/), Ferguson, et al. (U.S.Pat. Nos. 88,457,827, 8,504,233, 8,655,537, 8,676,430, 8,712,624,8,755,967, 8,825,259, 8,825,265, 8,831,813, 8,880,272, 9,026,300,9,120,484, 9,255,805, 9,261,879, 9,280,156, 9,310,804, 9,330,571,9,381,918, 9,459,625, 9,910,439, 9,932,035, and U.S. Ser. No.10/012,991), Zhu, et al., (U.S. Pat. Nos. 8,700,251, 9,216,737,9,315,192, 9,381,916, 9,495,874, and 9,766,626), Dolgov (U.S. Pat. Nos.9,120,485, 9,254,846, 9,381,917, and 9,561,797), Herbach, et al. (U.S.Pat. Nos. 9,523,984, 9,707,966, and 9,821,807), Litkouhi, et al. (U.S.Pat. No. 8,788,134), Dowdall (U.S. Pat. No. 9,336,436), Prokhorov, etal. (U.S. Pat. No. 9,573,592), Letwin, et al. (U.S. Pat. No. 9,580,080),Vallespi-Gonzalez (U.S. Pat. No. 9,672,446), Kato (U.S. Pat. No.9,896,101), Costa, et al. (U.S. Pat. No. 9,964,952), and Wang, et al.(U.S. Ser. No. 10/019,005).

FIG. 1 shows an autonomous vehicle (AV) incorporating systems andsubsystems that have been disclosed in the literature. The vehicleincludes:

-   -   (1) An AV controller 100 in communication with set of vehicle        control interfaces, including steering control 102; dashboard        with HCl (human-computer interface) controls 104; speed (brake,        acceleration) and transmission controls 106. The AV controller        may interface with a transceiver 101 facilitating        Vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure (V2I) and        Vehicle-to-Pedestrian (V2P) communications. A control bus 103        (i.e., CAN) is typically used to communicate with body and        vehicle controls, and a sensor bus 105 is used to interface with        various sensor sets as described below;    -   (2) A sensor set and associated hardware and software 110 for        tracking the vehicle's location in the world (over Earth) to a        several centimeter accuracy. This sensor set facilitates        self-location using one or more of lane marker detection, GPS,        3D point cloud measurements and matching, inertial measurements,        map-based correction and wheel or speed measurement;    -   (3) A sensor set for detecting obstacles and placing detections        into a world 3D model to support collision detection and        avoidance. This sensor set may include front-facing 120,        rear-facing 122, right rear 124, left rear 126 right        front/cross-traffic 128 and left front/cross traffic 130 video        and range sensors, which may include LADAR/LIDAR, RADAR, raining        or stereo computer vision, object recognition generally, image        learning algorithms, etc.;    -   (4) In terms of software, a planner generates paths from a        starting point to one or more destinations or intermediary        points employing terrain and/or road maps (graphs of encoded in        databases); and    -   (5) A path execution system generates vehicle controls (speeds,        steering, braking, signals, user or drive feedback like        speedometers, etc.) based on paths generated and data items        attached to the paths (from the maps) that indicate certain        behaviors in proximity to locations along the paths (i.e. stop        signs, speed limits, lane change prohibitions, or specific tasks        executed by AV payloads along the paths).

The AV systems and subsystems illustrated in FIG. 1 have been welldescribed in various references and other compendiums of open literatureprior art developed in the 1980 to the DARPA Urban Challenge in 2007.Exceptions, however, include the V2V/V2I, and V2P messaging interfaces.Whereas previous work has focused on specific behaviors or attributes ofthe single AV or how the single AV will behave relative to surroundinginfrastructure (i.e., intersections) and proximal/other vehicles sensedby its own internal set of sensors, there is an outstanding need to fuseexisting on-board AV systems with environmental interfaces to achieve asystem and method of autonomous transportation involving multiplevehicles and their surroundings, not just isolated AV control.

SUMMARY OF THE INVENTION

This invention improves upon the prior art by integrating autonomousvehicles—and non-autonomous vehicles—into a cohesive, interactiveenvironment, with communications to each other and to theirsurroundings, to improve traffic flow while reducing accidents and otherproblems.

Autonomous vehicles (AVs) and manually operated vehicles (MVs)send/receive messages to/from each other and from infrastructuredevices, enabling the AVs and MVs to determine the status of othervehicles, traffic conditions and infrastructure in their vicinity.“Status” in this context includes the current “state” of vehicles andsurrounding infrastructure, such as position, speed, orientation, andpossibly other factors such as turn rate, traffic directions,intersection conditions, blocked lanes, and so forth.

The AVs and MVs share common modules, enabling the vehicles to co-existin a fail-safe manner. The AVs and MVs store and operate in accordancewith a common set of rules based upon the messages received and otherinputs from sensors, databases, and so forth, to avoid obstacles andcollisions based upon current and, in some cases, future or predictedbehavior. Shared vehicle control interfaces enable the AVs to conform todriving activities that are legal, safe, and allowable on roadways. Suchactivities enable each AV to drive within safety margins, speed limits,on allowed or legal driving lanes and through allowed turns,intersections, mergers, lane changes, stops/starts, and so forth.

The above-described operation is achieved with various sensor sets andcontrol subsystems that adhere to path planning and executionguidelines. A self-location sensor set and tracking subsystems determinethe vehicle's location in the world (over Earth), preferably to anaccuracy in the centimeter range. Such sensor may include, withoutlimitation, GPS, 3D point cloud measurements and matching, inertialmeasurements, wheel or speed measures, and so forth as described herein.A sensor set is also provided for detecting obstacles and placingdetections into a world 3D model to support collision detection andavoidance. These sensors may include, without limitation, LADAR/LIDAR,RADAR, optical, ranging or stereo computer vision, object recognitiongenerally, image learning algorithms, and so forth as described herein.

A planner, operative to generate paths from a starting point to one ormore destinations, or intermediary points, employs terrain and/or roadmaps (graphs of encoded in databases). A path execution subsystemincludes generating vehicle controls (speeds, steering, braking,signals, user or drive feedback like speedometers, etc.) based on pathsgenerated. Data items attached to the paths (from the maps) indicatecertain behaviors in proximity to locations along the paths (i.e., stopsigns, speed limits, lane change prohibitions, or specific tasksexecuted by AV payloads along the paths).

Comprehensive communications capabilities provide messaging betweenproximate AV or manual vehicles, infrastructure, pedestrians and anyother active moving object in the driving environment. Suchcommunications capabilities may include, without limitation, thetransmission and/or reception of radio encodings and common messagepackets over common radio-frequency bands. Messages may includeinformation regarding status, location, speed, direction, and otherplanned intent to proximal vehicles, pedestrians and/or other movingentities or objects in the driving environment. Thus, in addition toanticipated or normal driving operation, on-board processing electronicsuses inputs from the sensor(s) and surroundings-to-vehicle informationto allow for lateral deviation from a planned path due to obstacles(other vehicles, pedestrians, or road obstacles large enough tointerfere with smooth driving) and road issues (construction, detours,lane shifts, etc.).

In the preferred embodiment, manually operated vehicles are providedwith some or all of the same subsystems used by the AVs to controldriving activities that are legal, safe, and allowable on roadways; thatis, to drive within safety margins, speed limits, on allowed or legaldriving lanes and through allowed turns, intersections, mergers, lanechanges, stops/starts, and so forth. Manually driven vehicles are alsoequipped with communications capabilities enabling messaging betweenproximate AV or manual vehicles, infrastructure, pedestrians and otherstationary/active moving object in the driving environment. As with theAVs, such communications capabilities may include, without limitation,the transmission and/or reception of radio encodings and common messagepackets over common radio-frequency bands. Messages may again includeinformation regarding status, location, speed, direction, and otherplanned intent to proximal vehicles, pedestrians and/or other movingentities or objects in the driving environment.

Broadly, vehicle control interfaces on both AVs and non-AVs imposeoperational behavior that is legal, safe, and allowable on the roadways,in a way which harmonizes with other manual vehicles, AVs, and thesurrounding environment, including infrastructure and pedestrians.Active traffic infrastructure elements are preferably provided toachieve this goal. Such elements may include hardware/softwaresupporting logic to implement traffic flow directives throughintersections, around work areas, over sections of the roadway or lanes,at specific points along the roadway, and the like. As withvehicle-to-vehicle communications, infrastructure-to-vehicle messagingmay use transmission/reception of radio encodings and message packetsover available radio-frequency bands. Optionally, active or passivelocalization and/or messaging tagging tags or devices may be worn orcarried by pedestrians or other roadway/vehicle protected objects inproximity to the roadway or driving environment for furthersafe/fail-safe operation.

The combinations of systematic controls support a wide range of drivingand transportation activities, including utility functions andnon-standard behaviors at stops or destination points. As examples, AVsmay identify, pick-up and place or drop persons or groups, as well asvarious payloads, including pallets, containers, packages, groceries,propane tanks, garbage, etc. AVs according to the invention may enter agarage or scan a parking area or street side, identify a parking spotand negotiate vehicle parking. Other capabilities include backing up totrailers, boats, and the like to affect coupling, towing, controlleddrop off and detachment. AVs according to the invention may furtheridentify a refueling device, automatically position relative thereto,couple to the refueling or charging fixture, detach and move off to thenext destination point.

Other supported AV operations may include at least the following:

Military applications including identification and positioning to anammunition, equipment or personnel reloading point, positioning, uploads(or downloads), and move off to a next destination point;

Entering, proceeding through, and exiting a washing station;

Execution of utility functions as an AV proceeds through its missionplan, cued by roadway locations;

Execution of behaviors cued by (a) passing by a point in the route planthat identifies beginning or ending of the utility function or behavior;(b) reception of a V2I message that identifies beginning or ending ofthe utility function or behavior; or (c) detection and decoding of avisual sign that identifies beginning or ending of the utility functionor behavior, including, without limitation:

-   -   i. Traffic controls and driving behaviors including speed limit        keeping, yielding, merging, stopping at intersections and        proceeding through them, entering and leaving traffic circles,        passing allowed/disallowed, turning left or right, traveling (at        reduced speeds) through construction zones, and u-turns, and    -   ii. Other functions concurrent with driving a route including        mapping, scanning for targets, (for military vehicles) firing on        targets, performing street maintenance or inspection (painting        lines, sweeping, snow plowing, dirt road grading, road surface        oiling or paving, or complex payload behaviors;

Utility functions or behaviors including proceeding through anintersection, inhibiting lane changes before the intersection for thevehicle stopping distance at the road speed limit;

Utility functions or behaviors including proceeding through an activelycontrolled intersection with a V2I intersection controller thatincludes:

-   -   i. Sensors to detect vehicles at the entrance or within to the        intersection and communicate this information to vehicles at and        inside of the intersection,    -   ii. Controls that manage lights sequencing from green to yellow        to red and associated other directions (lighted signs, lighted        director arrows, flashing red, flashing yellow, or other light        functions),    -   iii. Controls and communications to vehicles at the intersection        that disallow entry of vehicles into the intersection when one        or more vehicles are detected in the intersection, whether they        were signal to proceed by the intersection controller or entered        the intersection violating the entry protocol, and    -   iv. Controls and communications allowing specific vehicles at        intersection entries to proceed into the intersection and        proceed out one of a limited number of intersection exits;

Utility functions or behaviors including proceeding through an passivelycontrolled intersection between a higher priority roadway and a lowerpriority roadway, wherein:

-   -   i. Sensors in each vehicle at the entrance or within to the        intersection that detect other vehicles at or entering the        intersection and communicate this information to other vehicles        at and inside of the intersection,    -   ii. Controls and communications from each vehicle to other        vehicles at the intersection that disallow entry of vehicles        into the intersection when one or more vehicles are detected in        the intersection, whether they were signal to proceed by the        intersection controller or entered the intersection violating        the entry protocol,    -   iii. Controls and communications allowing specific vehicles at        intersection entries to proceed into the intersection and        proceed out one of a limited number of intersection exits        according to rules of the road, such as:        -   The vehicle passing through the intersection on the higher            priority roadway proceeds first,        -   A vehicle turning left from the high priority roadway            messages a request to enter across a high priority opposing            traffic lane and proceeds after the lane being crossed is            clear for at least the speed limit stopping distance plus            the time it takes to accelerate through the lane or it gets            a proceed reply from on-coming traffic,        -   A vehicle entering from the lower priority lane messages a            request to enter the high priority lane right and proceeds            when the right lane is clear for at least the speed limit            stopping distance plus the time it takes to accelerate            through the lane or it gets a proceed reply from on-coming            traffic, and        -   A vehicle entering from the lower priority lane messages a            request to enter the high priority lanes right and left and            proceeds left or passes through to the opposite intersection            exit when the right lane and the left lane are clear for at            least the speed limit stopping distance plus the time it            takes to accelerate through the lane or it gets a proceed            reply from cross traffic right and left;

Utility functions or behaviors including proceeding through an passivelycontrolled intersection between equal priority roadways (or an n-waystop), using one or more of the followings rules:

-   -   i. The first vehicle to the intersection entry proceeds first;    -   ii. If other vehicles are waiting to proceed, the one to right        of the vehicle that just passed through the intersection        proceeds next;    -   iii. If all vehicles are stopped at the intersection at the same        time, each waits a random time period and the vehicle that        computes the shortest time interval goes first, and then the one        to its right proceeds, and so forth;    -   iv. If the intersection is an n-way stop each vehicle must stop        before entering the intersection; and    -   v. Each vehicle at the intersection messages time when it is at        the intersection entry to the other vehicles present so a common        time base is established.

Merging onto a limited access roadway, which may include:

-   -   i. Message requesting merge, and    -   ii. Reply message acknowledging merge space available or        verification of space available by in-vehicle sensors;

Changing into an adjacent lane on a multiple lane roadway, which mayinclude:

-   -   i. Message requesting lane change, and    -   ii. Reply message acknowledging lane space available or        verification of space available by in-vehicle sensors;

Passing across the center line on a two way roadway for passing or leftturn, which may include:

-   -   i. Message requesting lane crossing, and    -   ii. Reply message acknowledging on-coming traffic gap available        or verification of space available by in-vehicle sensors;

Communications with a pedestrian wearing a location tracking device;

Communications with obstacle objects like pedestrians and other vehiclesto detect and predict the location of other obstacles by V2V and V2Pmessages redundantly with:

-   -   i. Obstacle detection sensors (LADAR, RADAR, Computer        vision-based object detection systems) to:    -   ii. Populate the world 3D map used to determine local paths        that (a) avoid obstacle, (b) stop before hitting the obstacles,        and (3) slow down to pace moving obstacles being followed;

Using polynomial path extrapolation to accurately project paths that aregenerated by Ackerman steered vehicles, assuming that linearextrapolation is insufficient;

Performing steering control based on inertial measurements and vehiclespeed measurements using data that is most accurate over a short timeframe, while correcting for these measures by over-watch sensors such asGPS or tracking based on measurements from the travel path proximal area(ranges measured to surrounding landmarks or relatively permanentfeatures of the environment, lane markers or lines on the roadway, roadshoulders, or other localized beacons or signs, and features); and

Evaluating potential collisions and/or collision avoidance behaviors bylimiting the obstacle search to the approximate lane width along itsintended path of travel or for moving objects that are travelling alongpaths that project through its line of travel in the presence of objectsare not dangerous and should be effectively ignored. The alternative ofbasing safety stop computations on a simple projection zone in front ofthe vehicle along a simple line will cause the AV to stop unexpectedly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that shows an autonomous vehicle (AV)incorporating systems and subsystems that have been disclosed in theliterature;

FIG. 2 illustrates a system according to the invention that combinesV2V, V2I, and V2P communication that integrates with, and augments,in-vehicle autonomous vehicle sensors, perception, and behaviors;

FIG. 3 depicts a software stack of autonomous modules, including a pathcontrol module that keeps the vehicle at a commanded speed, direction oftravel, and on a planned path, minimizing the difference between vehiclelocalization and the commanded path;

FIG. 4 shows behaviors TYPE, BEGIN, and END specified by a V2I messageemitted by road side element; and

FIG. 5 illustrates how V2P and V2P communications provide an alternativeway to detect collision/accident threats.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In parallel with the development of technology enabling AVs is the ideaof incorporating Vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure(V2I) and Vehicle-to-Pedestrian (V2P) transceivers. As envisioned by theUnited States Department of Transportation (DOT) IntelligentTransportation Systems Joint Program Office, short-range communicationssupporting high speed data exchange between vehicles can be the basis ofvehicle safety applications, including alerts to vehicles of imminenthazards and information such as:

-   -   veering close to the edge of the road    -   vehicles suddenly stopped ahead    -   collision paths during merging    -   the presence of nearby communications devices and vehicles    -   sharp curves or slippery patches of roadway ahead.¹        Further identified applications include:    -   Blind spot warnings    -   Forward collision warnings    -   Sudden braking ahead warnings    -   Do not pass warnings    -   Intersection collision avoidance and movement assistance    -   Approaching emergency vehicle warning    -   Vehicle safety inspection    -   Transit or emergency vehicle signal priority    -   Electronic parking and toll payments    -   Approaching construction    -   Detours    -   Narrow lanes    -   Shifting lanes    -   Commercial vehicle clearance and safety inspections    -   In-vehicle signing ¹ DOT, Office of the Assistant Secretary for        Research and Development, Intelligent Transportation Systems        Joint Program Office, DSRC: The Future of Safer Driving,        https://www.its.dot.gov/factsheets/dsrc_factsheet.htm    -   Rollover warning    -   Traffic and travel condition data to improve traveler        information and maintenance services

U.S. Pat. No. 9,952,054 [An, et al.], and disclosure 20180012492[Baldwin; Craig A., et al.] describe similar systems to implement thiskind of communication between vehicles and infrastructure, but do notexplain how this infrastructure is related to a complete realizablesystem for nearly 100% safe operation of autonomous vehicles (AVs) orautonomous driving systems (ADS).

While both basic AV technology and V2x infrastructures contribute tobetter safe future vehicles and roadways, the goal of true safety willnot be successfully achieved by more artificial intelligence or othertechnologies without a comprehensive, standardized AV vehicle androadway infrastructure architecture. Recent testing of AVs, whileshowing promise, indicates there appears to be a limit to the safety andfully automatic Level 4 or 5 automation as defined by the SAE.² Brandsom[July 2018]³ provides a strong argument that on its present course, AItechnology is not likely to achieve safe level 4/5 autonomy. A recentanalysis of disengagement statistics provided to California byPiekniewski [Sep. 2, 2018]⁴ suggests that present autonomous vehicle AImay be reaching a plateau, requiring a change in strategy to makecompletely safe AVs. Roadway system wide application of V2x technologystill awaits standardization and probably mandates from the DOT. ² SAE,“Taxonomy and Definitions for Terms Related to Driving AutomationSystems for On-Road Motor Vehicles”, J3016_201609, Sep. 30, 2018,https://www.sae.org/standards/content/j3016_201609/³ Brandom, Russell,“Self-driving cars are headed toward an Al roadblock,” The Verge, Jul.3, 2018,https://www.theverge.com/2018/7/3/17530232/self-driving-ai-winter-full-autonomy-waymo-tesla-uber.⁴Piekniewski, “Autonomous vehicle safety myths and facts, 2018 update,”https://blog.piekniewski.info/2018/02/09/a-v-safety-2018-update/.

To address these issues, we define a system (FIG. 2) that combines V2V,V2I, and V2P communication that integrates with, and augments,in-vehicle autonomous vehicle sensors, perception, and behaviors toprovide the following:

-   -   (a) Vehicles with at least doubly redundant capabilities for        detecting active traffic flow infrastructure (traffic lights,        speed limits, constructions zones, etc.);    -   (b) Vehicle with full knowledge of moving elements (other        vehicles and pedestrians) in the proximal driving area,        including future motion intentions out at least to the stopping        distance;    -   (c) Vehicles that are actively managed through actively        controlled traffic intersections (stop light intersections),        eliminating error-prone computer vision-based detection of        traffic lights and light state; and    -   (d) Vehicles that implement a common request/reply policy for        uncontrolled intersections and lane changes so that the traffic        can flow smoothly and fairly, allowing higher priority to        priority lanes, while providing holes in the traffic for vehicle        turning in from low priority streets into higher priority ones,        for lane changes, and ramps associated with highway merging.

The core to this implementation is a common set of shared vehiclecontrol behaviors which implement safe driving identically on everyvehicle (autonomous or manually driven), adding safety interfaces,limits and controls to manual vehicles so that the manual driver adheresto basic driving safety rules and regulations as rigorously as do theautonomously control vehicles, removes unreliable decoding of trafficcontrols, and provides for common standards of autonomy testing, sharingof vehicle behaviors that may evolve as the technology evolves, andcommon roadway regulation to achieve, ideally, 0% accidents.

Beginning with an autonomously driven vehicle (AV) or safe-manuallydriven (advanced driver assistive systems or ADAS) vehicle, we disclosea system that includes the components shown in FIG. 2. The core vehiclefunction is the AV itself, as shown in FIG. 1 but with—at a minimum:electronic controls for starting and stopping (engine or other powersource), speed control (throttle or electronic control of speed, andtransmission control), and steering. To this we add maximum stopping orbraking that is used when imminent collision is detected. Compared tonormal braking—a controlled deceleration rate for passenger comfort andplanned stopping distance, maximum braking is the fastest stoppingdistance possible associated with full vehicle braking action foremergency stopping, and maximum acceleration to avoid moving obstacles(such as oncoming speeding vehicles).

Over these basic vehicle controls is a software stack of autonomousmodules (FIG. 3). These applications may include a path control module(1) that keeps the vehicle at a commanded speed, direction of travel,and approximately on the planned path minimizing the difference betweenvehicle localization (2) and the commanded path. This module also willtypically perform planned slowing, stopping, or maneuver within anallowed lateral deviation from the planned path due to the detection ofobstacles (other vehicles, pedestrians, or road obstacles large enoughto interfere with smooth driving).

Over the path control module (1) is a local path planner (3) that keepsthe vehicle on its planned path, while at the same time, dynamicallymodifies the local path to avoid obstacles identified in the 3D workmodel. The world model (4) is populated by the suite of obstacle andterrain sensors, RADAR, LADAR/LIDAR, and/or computer vision processedfocal plane sensors). Over the path executor (or local planner) is aroute planner (5) that provides a connected set of path segmentsextracted from a map database (6) or graph that will take the vehiclefrom its pick-up or starting point to a specified destination. Over theroute planner is a mission segment planner (7) that defines destinationand intermediate destination points in order of visit and can also insome application, link to artificial intelligences that execute someform of work at some or all of the destination points. Refinementsinclude detection of road speed limits, determining if a slow vehiclecan be passed or must be paced (and whether passing is allowed or not),detection of space to enter a lane or turn into a circle or cross streetbased on detection of the positions of cross traffic or other obstacleslike pedestrians, and the state of active traffic controls (stop lightsor lighted traffic flow signs) or yield rules.

For many AVs, there is a utility function to be performed optionally ateach starting or stopping point in addition to moving from the startingpoint to the specified destination point. For a work vehicle⁵ this mightinclude accepting a load/unload location, acquiring and stowing a load,and/or unloading. Loads can be partial, or loads that completely fillthe storage capacity of a vehicle. For a taxi it will include stoppingfor passenger pick-up, accepting the passenger's method or payment, andaccepting the passenger's destination information. For a commercialtruck it might include coupling to a trailer, accepting a destination,and dropping the trailer at the destination. For a package delivery orgarbage pick-up system, substitute package/or other payload for personand perform similar pick-up and drop-off utility functions. For aconsumer vehicle it might include scanning a parking street side or areaor entering a parking structure or garage, identifying a space, andperforming a parking operation. For a vehicle that is trailer capable,it might entail lining up with a trailer and effecting trailer couplingand towing, or alternatively trailer back-up to a drop point anddetaching from the trailer. An AV might identify a refueling or chargingdevice, position to it or over it, couple to the refueling or chargingfixture, detach from the fixture, and move off to the next destinationpoint. For a military AV, a similar loading operation for ammunitionresupply might be a utility function. An AV might enter, proceedthrough, and exit a washing station or any other close, specialtymovement operation that requires non-standard behaviors. ⁵ In anautonomous material handling system as disclosed in U.S. Pat. No.8,965,561, Jacobus et al. destinations are associated with actions toinspect, pick-up, or place pallets or containers of goods.

The most complex aspects of behavior involve what the AV does as it isdriving though its planned routes from start points to destinations.These behaviors include specific driving behaviors such as speed limitkeeping, yielding, merging, stopping at intersections and proceedingthrough them, entering and leaving traffic circles, passingallowed/disallowed, turning left or right, traveling (at reduced speeds)through construction zones, and u-turns. However, it can also includecomplex payload behaviors such as mapping, scanning for targets, (formilitary vehicles) firing on targets, performing street maintenance orinspection (painting lines, sweeping, snow plowing, dirt road grading,road surface oiling or paving, etc.). These behaviors may be triggeredin following ways:

-   -   (1) Behavior TYPE, BEGIN, and END are specified as a property of        roadway segments defined in the AVs mission plan starting at        some waypoint and ending a some waypoint, typically either        specified in the mission plan or in properties of waypoints        defined in the map database;    -   (2) Behavior TYPE, BEGIN, and END are specified by a V2I message        emitted by a road side elements like that shown in FIG. 4. These        may emit standard format RF messages, flagging speed limit        changes, construction site barriers and markers, yield rules,        intersection controls, etc. For items like intersections that        are not static, and cannot be reliably detected visually, these        V2I messages are necessary for 100% safe control of traffic flow        through shared spaces (like intersections);    -   (3) Behavior TYPE, BEGIN, and END are determined by the AV local        perception system, such as presence/absence of other vehicles at        the entry points to intersections, visual road signs (for speed        limits, yields, passing allowed/disallowed, right or left turn        prohibitions, etc.), and street signs (marking entries, exits,        and naming or numbering roadway segments. These conditions are        always detected with a probability of correct detection        (typically in the 70%-95% range), and therefore wherever this        information is critical for safe vehicle operation, visual signs        have to augmented by devices that emitted the correct V2I        messages redundantly (i.e. case (2) above), or in AV carried or        downloaded digital maps (i.e. case (1) above).

Perhaps the most complex in-route behavior requiring V2V or vehicle totraffic control infrastructure collaboration is the actively controlledintersection shown in FIG. 4. Based on the messages defined laterherein, there may be many subsets of the core V2I transmitter shown inFIG. 4. For instance, a device to transmit speed limits or lane changelimitations might simply be a short-range signal transmitted without anysensing capabilities (8). However, at an actively controlledintersection controller (9) will have sensors to detect vehicles,controls to manage stop lights & signage, and its supporting V2Imessaging will be more complex, for instance defining the priority forentering and leaving the intersection that includes (a) yielding to anyvehicle violating the specified protocol, (b) proceeding when thetraffic control allows it, and (c) proceeding from an intersection entrylane to only one of the exit lanes allowed (i.e. turn/no-turnspecifications, right turn on red if it does not interfere with crosstraffic, if this is not disallowed, turning per specific active arrowsor signs).

In this disclosure, we place the responsibility for managing the in-flowand out-flow of vehicles with an updated wireless traffic controlelement that converts STOP, YIELD, TURN and GO commands presentlyencoded as physical lights (red, yellow, green, various illuminated turnarrows, and written information like “No Turn on Red,” etc.) intoelectronic commands communicated to all vehicles entering or waiting atthe intersection through V2I communications messages. Even manualvehicles will come to be equipped with on dashboard indicators that arecomplete enough to replace present day intersection traffic controllights (although during an extended phase in period, present day stop/golights in intersections will be retained for legacy vehicle traffic atthe expense of the potential for cross traffic accident events).

For uncontrolled intersections there is preferably a common protocolwherein vehicles on higher priority roadways take precedence over thoseentering from lower priority roadways. To cross or enter a high prioritylane, a gap in traffic must be detected that is at least the stoppingdistance of vehicles travelling at the speed limit plus the timerequired to accelerate from stop into or through the higher prioritylane crossed or turned into. As indicated below with regard to V2Vmessage sets, a the lower priority vehicle requests to enter to higherpriority roadway and receive an acknowledgement replay from on-coming orup-coming traffic before proceeding. For an intersection betweenroadways of equal priority, the first vehicle to the intersection goesfirst, unless multiple vehicles arrive simultaneously—then the one tothe right goes first.

However, in practice, human drivers often replace this protocol with onethat could be stated as the most aggressive goes first and the restaccommodate. Similarly, in merging into a lane (or into a circle), thecar entering is supposed to wait until a large enough space in crosstraffic allow safe entry. In practice again, however, in heavy trafficthe cross traffic must make space to allow entry or the entering drivermust aggressively force his vehicle into the cross traffic flow, orentry may not be possible in a timely manner. This is especiallyimportant on fast moving expressways where the incoming vehicle mustspeed up to match cross traffic flow, but then may not have a safelanding on the road shoulder if a space for safe entry is denied by thecross traffic. This invention is designed to accommodate these and otherreal-world situations.

Beyond difficult interception behaviors, V2P and V2P communications(FIG. 5) provide an alternative way to detect collision/accident threatsto (a) provide redundancy to commonly employed obstacle/collisiondetection sensors like RADAR,LADAR/LIDAR, and computer vision basedterrain and obstacle location, range, and closing velocity estimation;and (b) enable pedestrians to be part of the architecture so as tosubstantially reduce the likelihood of traffic fatalities due topedestrians entering a roadway from invisible or occluded locations,such as between parked cars, blind corners, or foliage blockingvisibility.

A ‘smart’ watch-like device (i.e., a carried smartphone, a smart pendantthat is worn, or and exercise bracelet) can integrate into the radionetwork (typically through cellular phone wireless) and can present thelocation of the person wearing or carrying the device, thus providingnearby vehicles location information that they would otherwise only beable to derive from obstacle detection range data. The obstacle data forany of the presently available sensors is strictly line-of-sight so manysituations are present where the autonomous vehicle cannot detectpotential collisions until after the victim is inside of the autonomousvehicle's stopping distance—an accident avoidance situation that isalmost impossible to mitigate. Furthermore, obstacles are typicallydetected only along the path an AV “intends” to drive, so on-comingobject only become obstacles if their path of travel intersects with theAV's planned path. Because the on-coming obstacle may “intend” to changepath toward collision, without that being directly measurable orextrapolated from obstacle track information, messages from vehicle tovehicle projecting their intended paths out to the stopping distanceeliminates the possibility of this confusion.

Any competent AV requires redundant sensors to detect obstacles,specifically vehicles, and objects that might enter the vehicle's drivepath as cross traffic (other vehicles and pedestrians). These might beRADARs looking in all potential travel directions, or alternativelyLADAR or computer vision-based object detection systems (also in everydirect of potential travel), but then also include V2V and V2Pcommunications means to collect data about current state and near futureone to several seconds of planned or predicted travel path and speed ofany vehicle or pedestrian in the near distance, i.e. within a one toseveral second radius at current vehicle-to vehicle orvehicle-to-pedestrian closing velocity).

For predictable roadway behaviors and ultimately roadway safety, allvehicles, including manually driven ones, have to obey a core set ofcommon driving rules which we summarize in the following set of rulesand procedures. These rules should be fully disseminated through acommon documentation process which mirrors the RFCs that compose commonInternet Protocol,⁶ and openly archived and distributed to anymanufacturer of equipment or vehicles that are to be allowed inside ofany roadway system that is designated as autonomous capable. As the AVrules are refined, corrected, and extended, the new changes also need tobe openly archived and distributed to any manufacturer of equipment orvehicles so the entire system can stay correlated. The issue will bewhether these rules can be privately maintained (ex: like the InternetEngineering Task Force), maintained by a worldwide standards body, or byindividual national or state governments. That will be determined byconvention, legislation or other political action which is beyond thisdisclosure. ⁶ RFC: Request For Comment—maintained for the Internet byInternet Engineering Task Force, https://www.rfc-editor.org/retrieve/

The following summarizes the implementation of an AV architecturethrough rules, data achieving, and necessary error correction:

Localization

The core subsystem to support managed autonomy is maintaining accurateknowledge in each active vehicle of where it is (and its dimensions) inthe managed transportation area (roadway, parking area, etc.). Howaccurately position information can be maintained defines how close toeach other vehicles and other roadway objects can be allowed to operate.For instance, if we could maintain a 2 cm localization accuracy, at lowspeeds it is possible to come within that distance and a safety factor(may be two times) to adjacent objects without collision risk. Practicalminimum accuracy for roadway vehicles is about one tire width or about8″ (˜200 mm). Most precision GPS localizers can meet or better thisrequirement with 6 satellite fixes. Inertial measurements exceed thisaccuracy until random-walk errors build up. This suggests that foroutdoor localization, employing precision GPS to bound localizationerror (i.e. to bound random walk errors from the internal sensors),combined with inertial updates to provide faster smoother updatedinterpolations, is an ideal localization solution.

In addition, it must be possible to extrapolate future path of vehicles(for example, 1-6 seconds out) based on planned path, speeds, andsteering angles. For simple path projection from a list of priorlocation points, a second order polynomial of the following form workswell because it captures the fact that over a short time interval whenthe steering angle is effectively fixed, location track points willexactly fit this order of curve though out a straight and turning pathsegment (some people have used linear extrapolators which will provideaccurate projections over very short time intervals or straight paths,but do not support longer projections through curves or steeringoperations).

One second-order overwatch extrapolator constructs two 2^(nd) orderpolynomials for the most recent AV location measurements (fused or froman overwatch sensor like GPS or point cloud localization). Onepolynomial for X and one for Y, with time being the independentvariable:

P _(x)(t)=Ax*t*t+Bx*t+Cx  Eq. 1:

P _(y)(t)=Ay*t*t+By*t+Cy  Eq.2:

The coefficients are calculated using linear least squares regressionfrom a prior list (prior n localizations, perhaps 1-6 seconds worth ofdata). The derivatives, Vx(t) and Vy(t), are calculated for the midpointof the temporal envelope for X and for Y by:

V _(x)(t)=2*Ax*t+Bx  Eq. 3:

V _(y)(t)=2*Ay*t+By  Eq. 4:

Heading, α, is calculated as α=a tan 2(Vy, Vx), and compared against theestimate of heading at that time (a estimate). The difference betweenthe α and α estimate is then applied to smooth the current estimate frominertial measurement.

Maneuver to Move from a Starting Point to Destination Points:

The core function of an AV is to transport payloads and/or people from astarting point to a destination point. This is accomplished by locatingthe starting point on or near a digital roadway map. Then search isperformed to identify an acceptable path from this starting point to thedestination, also located on the map, that minimizes cost metricsincluding minimum distance, time, safety, fuel use, fewest congestionsegments (segments where traffic congestion is reported), and maximumutilization of priority segments (segments with larger traffic flow andhigher speed limits). Typically the path search utilizes an A* typesearch⁷ algorithm which generates best paths limited by a heuristicsearch computation limitation. ⁷ Zeng, W.; Church, R. L. (2009).“Finding shortest paths on real road networks: the case for A*”.International Journal of Geographical Information Science. 23 (4):531-543. dol:10.1080/13658810801949850

However, maps serve another important function in an allweather/conditions capable AV. While there are demonstrated lane keepingsensors, these are typically only good enough for well manicuredhighways (where lane markers are very well maintained and visible tosensors) during good weather conditions (no snow, dirt, rain, etc.obscuring the lane markers). Therefore, an all-weather AV must employother lane tracking techniques to augment lane detection throughsensors. The proven method is to use a combination of the precisiontracks encoded into the digital map and the vehicle localization system.

The localization system described above provides vehicle location towithin better than a tire width. If the digital tracks in the map arealso better than that, lane keeping can be accomplished through asteering servo designed to keep the vehicle on the planned track withinthe localization sensor error.⁸ Speed limits and static traffic controls(yield, speed limits, lane closures, stops, right left turn limitationsat intersections, etc.) can also be encoded into the map. The core ofthese features was demonstrated in 2007 at the DARPA Urban Challenge,⁹where roadmaps were encoded as RNDF graphs (Route Network DefinitionFiles).¹⁰ From starting points to destination points were encoded as MDFfiles (Mission Definition Files)¹¹ that specified starts, intermediatedestinations, and final destination over an RNDF road network. Whilethese files define road graphs, these can be coded in a computer in manyways, including as relational databases, directed graphs, semantic nets,array lists, lists, associative memories, etc. The encoding simplyaffects how the data is searched to find paths. ⁸ Snider, Jarrod M.“Automatic steering methods for autonomous automobile path tracking.”Robotics Institute, Pittsburgh, Pa., Tech. Rep. CMU-RITR-09-08 (2009).⁹DARPA, Urban Challenge, https://www.grandchallenge.org/grandchallenge/¹⁰DARPA, Route Network Definition File (RNDF) and Mission Data File (MDF)Format, Mar. 14, 2007,https://www.grandchallenge.org/grandchallenge/docs/RNDF_MDF_Formats_031407.pdf.¹¹DARPA, Route Network Definition File (RNDF) and Mission Data File (MDF)Format, Mar. 14, 2007,https://www.grandchallenge.org/grandchallenge/docs/RNDF_MDF_Formats_031407.pdf.

Path planning derived from the graph network is a 2-stage process. Thefirst stage uses a spatially constrained A* process on a gridconstructed from known map data plus currently known obstacles. Thisprovides a course path from the vehicle start position to a targetdestination. The second stage uses a 3-degree of freedom (x, y, heading)Rapidly-exploring Random Tree (RRT) algorithm¹² constrained by steppingalong the A-star path. Each “edge”/step of the RRT is a turn, followedby a straight drive, followed by a turn. The RRT explores the space fora configurable time and returns the best solution for each step thatprecludes paths that exceed the turning radius of the vehicle. Some formof constraints to accommodate vehicle turning, braking, and accelerationlimitations are required so that proposed paths derived from the mapdata are executable by the target vehicle. LaValle, Steven M.; KuffnerJr., James J. (2001). “Randomized Kinodynamic Planning” (PDF). TheInternational Journal of Robotics Research (IJRR). 20 (5): 378-400.doi:10.1177/02783540122067453.

Implementation of V2V, V2I, and V2P Radio Messaging:

-   -   1. Standardized V2V and V2I Radio Frequency parameters, data        grams, and content. Message grams encode the following:        -   a. Plan “where entity is” (GPS coordinates—about 6 seconds            of plan data depending on travel speed)        -   b. Plan direction and rate of speed (6 seconds of plan data            depending on travel speed)        -   c. Actual where, direction, turn rate, and speed        -   d. Status—vehicles, functional status (good, degraded,            failed) (occupied/not occupied) (manual drive/automated            drive, supervised/unsupervised automated drive)        -   e. Status—infrastructure—lane blockage, prescribed            mitigation: merge left or merge right        -   f. Status—lane changes/passing allowed right, allowed left        -   g. Status—infrastructure—Intersection            -   i. Encode incoming and outgoing for each lane            -   ii. Encode lane properties straight only lane, straight                or turn right lane, straight or turn left lane, right                only lane, left only lane            -   iii. Encode this lane yields to cross traffic, cross                traffic yields to this lane            -   iv. Encode lane active regulation, red (stop), yellow                (pending red stop), green (go)        -   h. Status—infrastructure—speed limits (min max)    -   2. Messages for client server navigation and status reporting        -   a. Messages to down load map segments for route planning to            vehicles        -   b. Messages to down load routes if server performed route            planning for the vehicle        -   c. Messages for municipalities or other Government to upload            changes in the roadway (new roads, closed roads, work areas,            etc.)        -   d. Messages for work crews to to upload changes in the            roadway due to road maintenances or construction        -   e. Messages for vehicles and vehicle operators to upload            status changes including maximum, minimum, and average drive            speeds, debris on the road, adverse driving conditions            reports, locations in need of roadway repair, road            blockages, etc.        -   f. Messages from server to clients to effect dynamic traffic            flow—includes reprogramming or intersection light            priorities, speed limits lowered or raised due to road            conditions, messages that suggest route replanning due            changes in traffic flow or road conditions.        -   g. Messages for support of differential GPS (including            indicating that Differential GPS corrections are            unavailable).

Defining AV Standard Cooperative Driving Methods, Behaviors:

-   -   3. Standardize lane keeping, merging and intersection driving        behaviors:        -   a. No passing within x distance to intersection (x set based            on stopping distance at roadway travel speed).        -   b. No lane change or merging, if lead vehicle is less than y            distance ahead at x travel speed (x set by lead vehicle            merging vehicle speeds—a 70 MPH x should be about 7 car            lengths of approximately or 4-4.5 meters*7, approximately 28            meters) and traveling faster or equal to the merging vehicle        -   c. No lane change or lane merging, if side or trailing            vehicle less than y distance (y also set as MPH/10 car            lengths as in b above) behind and travelling equal to or            slower than the merging vehicle speed; note message            requesting merge, and response from the trailing vehicle            should occur before the change or merging so that in dense            traffic this condition can be created for the in-merging            vehicle.        -   d. No entering of an unmanaged intersection unless cross            traffic vehicle less that y distance away (y set            approximately as cross traffic MPH/10*car lengths*2 to            account for acceleration rate from near 0 MPH).        -   e. Enter un-managed intersection according to right of way            rules. First to intersection goes first. If two or more at            the same time (or cued) right most has right of way—goes            first        -   f. If above rules don't apply (i.e. no vehicle can determine            that it has entered first and vehicles are waiting for a            turn to enter the intersection), execute a randomized wait            time then go if no other vehicle has entered the            intersection within the wait time.        -   g. In intersection, conflict generates stop until resolved            (i.e. stop if anyone is in the intersection already, wait            for them the clear through)        -   h. Maintain safe head distance of y distance at x speed for            safe stopping (this is actually a table relating stopping            distance to travel speed)            Defining Methods for Integrating Both Manual Driven Vehicles            into the Same Roadway as AVs:    -   4. Autonomy and Manual vehicle integration        -   a. Whenever driving unsupervised (or supervised by a driver            with hands off the wheel) provide a standardized visual            signal visible to other drivers that shows operation in an            automated mode (Flashing yellow light?)        -   b. Standardize the means for supervising driver to take            control from an automated driver (tap brakes, grab wheel,            etc.)        -   c. Per Safety 2.0: An Autonomous Driving System (ADS) must            provide a standardized means for informing the human            operator or occupant through various indicators that the ADS            is: Functioning properly, currently engaged in ADS mode (on            or off), currently “unavailable” for use, experiencing a            malfunction; and/or requesting control transition from the            ADS to the operator.        -   d. Standardize automated driver signaling to the supervising            driver that control should be assumed due to safety or            failure mode conditions        -   e. If faults are not attended to by the supervising driver,            use a standard visual signaling means to indicate automated            failure mitigation procedure is in effect. Best mitigation:            Move right to shoulder and slow to stop if systems generate            or detect faults unless positive control is assured from            supervisory driver. Next best mitigation: If moving to the            shoulder is not safely possible slow to stop if systems            generate or detect faults unless positive control is assured            from supervisory driver.        -   f. Apply turn signals, brake lights, and headlights in the            standard way to respectively indicate turns and lane changes            in advance of the action, braking effort when applied, and            at dusk (unless always on)        -   g. Define a standard dash display to indicate stop/go (red,            yellow, green) status at managed intersections displayed to            each drive applicable to his/her lane and incorporated into            both manually and automated vehicles from some date forward.            This is an in-vehicle rendition of the data which presently            would be displayed by stop light assemblies, but without the            subjectivity of what traffic control applies to which            vehicle at the intersection.        -   h. Define a standard for automated and manual vehicles to            maintain posted speed limit maximums from some date forward.            Can be by encoded digital map or by RF active speed limit            postings but also likely to be updated immediately through            V2I messaging.        -   i. Define a standard for assuring adequate safe stopping            head room for automated and manual vehicles to maintain            posted speed limit maximums from some date forward. This is            an internal speed limited based on forward vehicle detection            and travel speed/conditions.        -   j. Define a standard for a device that can be carried by            non-vehicles (perhaps from a pedestrian carried cellular            phone, etc.) that provides P2V data comparable to x above            for non-vehicle (pedestrian) so that all vehicles, automated            or manual can reliably detect and plan for pedestrian            actions around the roadway. This prevents pedestrians            entering into the not-able-to-stop zone in front of            vehicles—vehicle behavior would be to slow or stop yielding            to the pedestrian before the zone is entered.        -   k. Provide a standard for disabling pedestrian tracking            device messaging when the pedestrian enters into a vehicle.            The Common Means for Testing that all AV Architecture            Compliant Vehicles (Manual and Autonomous) are Rules            Compliant:    -   5. Testing and behavior assurance        -   a. Define a standard means for testing vehicle products to            these standards to assure design reliability.            -   i. Include design procedure guidelines            -   ii. OEM testing guidelines            -   iii. Certified National specific testing Lab(s)            -   iv. Standards for safety non-volatile logging at a                minimum including: recording of vehicle status,                position, speed, direction data, manual driver automated                driver controls state (for instance, steering, braking,                speed control actions), basically the same parameters                communication in V2V minimum message set.        -   b. Maintain an open source National incident reporting            system for accidents, causes, effects, mitigations—this            includes all automated behaviors required to meet the            National standard as it is initially defined and evolves            through identification of accident causing faults and the            mitigation behavior determined to resolve/remove these            faults.        -   c. Create a standard training course and certification for            accident scene investigators—link back to the National            incident reporting database so that accident incident study            results are captured and analysis put into the common            incident knowledge base.

V2V, V2I, and V2P Radio Messaging Design and Functions

A common communication standard for V2x, including V2V, V2I and V2P/P2V(vehicle to pedestrian and pedestrian to vehicle) must be implementedinto a autonomous transportation system infrastructure designed to takethe likelihood of accidents to near 0%. Most accidents in a human driventransportation system are due to human error or misjudgment. To get tonear 0% it is necessary to take all the guess work out of the system,replacing it with accurate predictive information. At the core of thiseach active participant must communicate intentions into the future fora long enough period of time to support accommodation by other activeparticipants, and/or detection of plan errors so the active participantcan reliably re-plan for safe operations. The typical stopping distanceof a passenger vehicle is estimated to be Speed+Speed/2=Number*3 andthat is the braking distance. At 70 MPH, 70 MPH+35 MPH=105 ft*3=315 ftstopping distance. Travelling at 70 mph, 315 ft is equal to315/102.7=3.1 seconds (approx).¹³ ¹³ A more complex estimate of brakingdistance is described in https://en.wikipedia.org/wiki/Braking_distanceand many tables have been composed to provide estimates.

Given that traffic might be heading in two directions, a safe time mightbe double that or for 70 MPH 6 seconds. The same calculation at 30 MPHyields 135 ft but still 3.1 seconds. Thus, each vehicle in a trafficproximity (defined as the worst case stopping distance at vehicle toobject closing speeds) needs to be aware of the future drive path of theother vehicles in the same proximity. This implies that V2Vtransmissions should include between approximately 6 seconds of futureplanned driving path (plan location, speed and steer angle) so all othervehicles in the same zone can avoid. Furthermore, travel velocity for avehicle has to be reduced to stay within the zone where all other pathknowledge is available and accurate.

Pedestrians move slowly (relative to vehicles) within the proximityzone, but are easily obscured by fixed and other moving objects (parkedcars, shrubbery, etc.). Therefore they also must project their futureintensions within the speed limit determined proximity distance.Therefore a pedestrian's future location based on projection of path,direction, and speed, should be projected about 6 seconds as well (alongwith an assumption by vehicles of a 6 second zone of random walkuncertainty because he/she could readily change directions of travel).This requires a V2P communication protocol to compute pedestrian travellocation, rate, and direction into a projected path for consumption byvehicles in the same proximity. This may be accomplished with an activetagging system like that in the Apple iWatch (GPS & LTE datacommunications) or a comparable device or pendant.

The pedestrian tagging device must disable messaging when the pedestrianenters a vehicle so as not to interfere with V2V communications. Thiscan be accomplished many ways including detecting in-vehicle emissions,for instance from automated ignition key fob; detecting vehicle positionand pedestrian position information as being essentially the same(overlapping in position); detection of the vehicle's wireless phoneconnection network (typically Bluetooth), etc.

Since infrastructure does not move, it need not participate intransmissions that communicate positions or tracks. However,infrastructure at a minimum needs to communicate speed limits (min/max)or lane limitations (i.e. lane blockages or re-routings), and allowedtravel through controlled intersections (stoplights). Static informationlike speeds and lane closures could be communication through the AV'sstatic roadmap databased, and not through V2I messaging, however, V2Imessaging provides a means for immediately updating data overridingstatic information in a down loaded map.

Decoding the infrastructure information through computer vision or othersensors of traffic flow through intersections is no completely reliable(a recent paper by Fairchild at al. cites a probability of correctnessof 95%-99% which would translate into worst case 1 errors per 20 trafficcontrolled intersections—clearly not 100% assured safety).¹⁴ In ourapproach, the traffic signals would transmit intersection status to allthe vehicles entering into the intersection proximity. Each lane will be(a) blocked, or (b) free to allow vehicle movement. Lanes allowingmovement into the intersection will offer list of exit lanes allowed tothe vehicle entering the intersection from the entry. This, forinstance, might offer the vehicle in the right hand lane the option toproceed straight through the intersection, or to take a right turn, butmight not allow a left turn if this is prohibited. Such a syntax willall coding of all allowed traffic directors through wireless messagesthat presenting are present to human drivers as turn arrows, left/rightturn prohibited signage, light colors (red-stop, yellow prepare to stop,and green proceed), etc. ¹⁴ David I. Ferguson, Nathaniel Fairfield,Anthony Levandowski, U.S. Pat. No. 9,145,140 B2, Robust method fordetecting traffic signals and their associated states, also described inFairfield, Nathaniel, and Chris Urmson. “Traffic light mapping anddetection.” Robotics and Automation (ICRA), 2011 IEEE InternationalConference on. IEEE, 2011.

Vehicles themselves will accomplish some of the function of trafficcontrol infrastructure at intersections where this infrastructure is notpresent (uncontrolled intersections). Most state driving regulationsstate that a vehicle on a higher priority route has precedence overvehicles turning from a lower priority route into the higher priorityroute. The issue is determination of route priority. This can beaccomplished either through active infrastructure messages or through apriori knowledge derived from a digital map onboard each vehicle. Atuncontrolled intersections of equal priority routes, most state'sregulations specify that the first vehicle to the intersection haspriority and all others are supposed to yield.

When two vehicles arrive or are at the intersection stopped at the sametime, the one to the right has priority and the rest are supposed toyield. Once one vehicle goes through the intersections, the one to itsleft would go next in a round robin style. As anyone who drives knows,human drivers do not strictly adhere to these rules so to support anautonomous vehicle infrastructure the rules have to become embedded intoto vehicle's automated controls and communicated as intentions to moveto all the other vehicles in the intersection proximity—disallowingillegal driver prematurely ceasing priority out of turn. Otherwisepolite automated vehicles will get stuck or will slow down trafficthrough the intersection due to aggressive manual drivers.

When entering into a yield intersection (like a roundabout where theentering vehicle yields to those in the circle, or onto a freeway wherethe entering vehicle yields to traffic already on the freeway), messagesrequesting entry into traffic from the yielding vehicle must be detectedby traffic that has priority or the incoming vehicle may not ever beable to enter the priority traffic flow, thus completely blocking feedtraffic flow. The traffic in the priority flow has to detect the requestto enter message, slow sufficiently to make a “hole” in the traffic andthe yielding vehicle has to speed up and enter that hole when itarrives. In all the autonomous systems presently under testing, this isaccomplished through simply waiting for the hole which may never arrive.In a production autonomous transportation system that is designed tohandle heavy traffic, the requesting of and creation of entry holes willhave to be accomplish through an active request message and responsebehavior (and response message indication the availability of thecreated entry hole).

Messages to and from Navigation and Roadway Status Servers

In some map and navigation services, large maps are downloaded into avehicle navigational system during service or when software updatesmight occur. However, modern systems employ periodic wide area networkincremental updating, generally employing cellular wireless internet ora proximal WiFi digital network connect.¹⁵ This technical approachallows for continuous update from the vehicle to the navigational serverof items including new route requests, maximum, minimum, and averagedrive speeds being driven, debris on the road, adverse drivingconditions reports, locations in need of roadway repair, road blockages,etc. It also supports downloading to the vehicle (the client) from theserver data like new map segments for route planning, suggested routesif server performed route planning for the vehicle, changes in theroadway such as new roads, closed roads, construction work areas due tonew construction or maintenance, support messages for differential GPScorrections, and/o dynamic traffic information like heavy trafficstoppage, changes in speed limits, intersection light or flow prioritychanges, etc. Because these messages are generated from any vehicle onthe road and each vehicle must access a common set of road, speed limit,traffic, and blockage data to assure that they all behave according to acommon set of rules over a common set of input navigation planning data,a minimum set of messages describing each item cited must be define withour system. ¹⁵ Google Maps, https://en.wikipedia.org.wiki/Google_Maps,Oct. 10, 2018.

Presently for “optional” GPS navigation aids to human drivers, eachvehicle can acquire such data to the extent that it is available fromalternative vendors (or not at all for vehicles that are operatedwithout the aid of GPS navigational aids). Moving over to a systemevolving to fully autonomous behaviors, this utilization of differentand not necessarily correlated data sources for plan will cause unsafeconditions (for instance, when one vehicle sees an upcoming work areaand begins to take appropriate speed and evasive maneuvers, whileanother vehicle, not sharing this information, does not). While theconcept presently implemented by client server systems like Google Mapsdoes not have to change fundamentally, it does become necessary todefine a open protocol that shared across the vehicle fleet and isaccessible by municipalities and maintenance work crews so all vehiclesshare a common knowledge base describing the driving environment. Alsoit is worth noting that present fully autonomous vehicles employ mapswith resolution down to the road lane, and approximately +/−2-10 cm (notlarger than approximate a tire width). This resolution of map data,while able to be acquired and available in some roadway areas, exceedsthe present resolution of data distributed by client-server systemspresently deploy for driver directions quality GPS navigation systems(Google Maps, Mapquest, Tom-Tom, Waze app, etc.).

Describing Cooperative Driving Methods, and Behaviors

The majority of AV driving behavior is to safely keep to the planneddriving path, within a given lane of traffic. However, more complexbehavior becomes necessary to change lanes, merge unto highways,navigate through intersections, and make smaller navigationaladjustments to avoid obstacles and sometimes other vehicles. With theV2V capabilities already described, most vehicles, pedestrians, andinfrastructure imposed rules and conditions can be planned for with aproximal area around a vehicle in motion. Data derived from thesemessages is augmented by direct obstacle measurements from RADAR, LADARand video captured and processed sensors. The basic process is thatmoment by moment, the vehicle creates a future planned path (overdistances necessary to create the status messages previouslydescribed—typically 1-6 seconds worth), and then intersects that pathwith a known obstacle map locally centered over the vehicle out toranges within that 6 second distance interval (at 70 MPH, approximately600 feet).

If the planned path takes the vehicle inside the safe distance from anobstacle (at high speeds nominally 1-2 meters, at lower speeds perhapsas low as 12-15 cm), the path is modified to produce a larger safetydistance. If the path deviation exceeds a preplanned maximum (so in anexpressway lane which is approximately 12 feet (3.7 m), with a vehiclewhich is about 2 meters wide and lane centered, the allowed pathdeviation might be set to 3.7-2/2 meters or 1.65 meters), the vehiclemust employ slowing or stopping (i.e. slowing to the estimated speed ofthe obstacle which is stopping for a fixed obstacle or speed pacing fora moving vehicle blocking the lane ahead).

Considering the roadway leading to an intersection as a zone wherepassing is disallowed, the no passing distance is at least the stoppingdistance at speed (which we previously estimated as 3 seconds of travelso at 30 MPH that is 135 feet, at 70 MPH, 315 feet).

To merge into a roadway with moving traffic, a hole sufficient for safeentry has to be made. This should be requested by the vehicle merginginto traffic and acknowledged by cross traffic vehicles. Vehicles incross traffic acknowledging the request should slow down to create asufficiently sized hole. This behavior applies to lane changes, trafficcircles, un-managed intersections where cross traffic has priority anddoes not have to stop, and for merging onto highways from entry ramps.Merging onto highways assumes that the entry vehicle is accelerating upto the traffic speed so spacing obeys highway spacing rules (one carlength per 10 MPH between cars so the space needs to be that times 2).Entering into an unmanaged intersection with through cross trafficrequires at least similar same spacing to allow for the time needed toaccelerate from zero to the cross traffic speed.

In general, automated vehicles should avoid lane changing, but sometimesthis is unavoidable due to an up-coming left turn, stalled vehicles inone lane, etc. Lane changing is s special case of crossing or enteringcross traffic. The vehicle changing lanes should send a lane enterrequest to proximal vehicles. It should measure the traffic hole in theadjacent lane and inhibit lane change unless the hole is greater than orequal to one car length per 10 MPH*2 between leading and trailing cars.Lane changes should be inhibited into intersections for nominally onecar length per 10 MPH from the intersection.

Managed or stoplight intersections can be built almost arbitrarilycomplex with right or left turn only lanes, pop-up messages like “NoTurn on Red,” and multiple RYG lights located at corners, over specificlanes, and orientated horizontally or vertically. The vast array ofalternative in operation in North America call for active trafficcontrol through managed intersections via V2I messaging as discussedearlier. Under this model, visual decoding of lights and traffic flowcontrols is replaced by V2I messages that direct traffic through theintersection according to the intersection-specific rules (which aremarked on the roadway and presented to manual drivers through visualcues). We suggest that future manually driven vehicles are to beequipped with dash-mounted stop, go, warn, and turn director displaysthat present the digital traffic flow riles applicable to the specificvehicle/drive based on the same V2I messages which are sent to automatedvehicles.

Unmanaged intersections, where one set of through traffic does not stop(has priority), is handled as indicated previously with low prioritytraffic stopping until the higher priority traffic either abates, or awhere a traffic hole of sufficient length or time interval has beencreated as a consequence of V2V messaging from a low priority vehicle toproximal higher priority vehicles requesting creation of a hole intraffic flow sufficient for the lower priority vehicle to enter thehigher priority traffic flow.

For n-way stop unmanaged intersections (which is what any unmarkedintersection should be consider by default), each approaching vehiclestops. The vehicle which is measured to be the first to stop, gets thefirst right-of-way to proceed through the intersection. Then around theintersection counter clockwise, the next vehicle (to the right) goesnext. If two vehicles arrive simultaneously, the one to the left yieldsto the one on the right. As sometimes happens due to timing, judgment,or system failure, if vehicle populate all entries into the intersectionsimultaneously, any one of them could go first (the rest yielding) tostart alternation through the intersection. A simple method fordetermining which vehicle will go first is through employing vehicleembedded random number generators that translate into wait time beforetaking the initiative to go. When any of the vehicles goes first, therest follow the yield right rule to resolve who goes next. If for somereason two vehicles go simultaneously, they should proceed through theintersection with full obstacle avoidance behavior enabled so anyprospect of collision is mitigated.

Transition from all Manually Drive Vehicles to Mix Manual AutomatedVehicle Traffic

Regardless of how one implements autonomous driving it will be performedby algorithmic methods that model vehiclephysics—acceleration/deceleration rates acceptable to passengers,maximum stopping for avoiding dangerous potential collisions, vehiclepower and mass functions projected forward to avoid collisions, etc. Itwill not involve modeling of detailed assessment of surrounding driverpsychological behavior to the degree that a human driver might assess.For that reason, AVs are likely to require more rigorous enforcement ofvehicle spacing, speed limit obedience to the letter, and a propensityto stop when obstacle are presented suddenly. Thus, in some ways, the AVautonomous driver will more like an inexperienced student driver than amore mature driver. We mark student drivers with a warning sign. We markwork vehicles like forklifts or earth movers that might present a dangerwith audible tones and flashing lights. We mark extra wide loads withsigns and lights. Similarly we will require rules that standardizemarkings to show when an AV is driving in autonomous mode. Similarly theADS should employ all manual signaling means (brake lights, turnsignals, back-up lights, headlights, etc.) in the same way as would ahuman driver.

As disclosed earlier, an AV integrates the ADS and manual drivingenvironment, manages intersections and active traffic controls, andcommunicates with vehicles through V2I messaging. Both manually drivenand ADS driven vehicles should provide a display that indicates go,stop, and warn stop light status, and means to indicate lane turninstructions, individualized to each vehicle entering the intersectionby lane. Similarly, speed limit messages should be detected anddisplayed on the user interface (as well as enforced by vehicle speedcontrols), eliminating the possibility of exceeding speed limits.Because the vehicle will know its speed and speed limit at all times, itshould enforce, even on manual drivers, safe stopping room to thevehicle ahead (1 car length per 10 MPH of travel speed).

In dense urban areas, depending on sensing and measuring the distance toa pedestrian and avoiding such obstacles may not be adequate. Both themanual vehicle and the ADS require a means to detect and localizepedestrians in the same way as they will be able to detect and localizeother vehicles. A pedestrian worn device like an Apple iWatch includes aGPS locator and the means for entering into the local cellular phoneradio network (which is or potentially is part of the V2V communicationmesh). This such a device allows on-coming vehicles to localizepedestrians even if they are obscured by infrastructure, parked cars, orother obstructions—eliminating surprises to an ADS or manual driver thatoccur when the pedestrian “pops” out from behind something.

Because any anticipated AV autonomous driver system cannot beanticipated to be perfect (i.e. to have knowledge of how to control thevehicle under all possible roadway, weather, and driving situations), amanual driver will require an industry wide standard approach toreacquiring control. Cruise controls employ either manually turning offcruise or tapping the brake to stow down the vehicle. These methodswould be comparably applicable to taking a vehicle from automated modeto manual mode. However, manual re-acquisition of steering will benecessary as well so to implement these methods, we would add detectionof re-acquisition of steering through wheel grasping and effectingmanual steering contrary to any automated steering torques (i.e. thewheel tracks an external application of steering force over anyinternally generated force from an autonomous steering control system).

While in autonomous mode, or under the control of an ADS, the ADSdisplay to the manual driver must include that the ADS is functioningproperly, the status of ADS engagement (ADS is on or off), that the ADScurrently “unavailable” for use (may be due to sensors malfunction,autonomous controls malfunction, weather or road conditions outside ofnominal, a roadway or map segment which precludes or prohibits ADSoperation, etc.), and the means requesting control transition from theADS to the human driver or operator (standard means for signaling thedriver to take control from the ADS—flashing visual indicator, vibratingseat, vibrating steering wheel, verbal commands, etc.). If the driverdoes not attend to this request for control transition within apredetermined time (1 to 2 seconds), the ADS must provide a behaviorthat effectively safes the vehicle—typically this would be slowing andstopping, and where the road provides a shoulder, steering over on toit.

Testing and Dissemination of Policies, Procedures, and Algorithms

Like the Internet of information we now take for granted, a network ofautomated vehicles will require a base layer of behaviors that arecorrectly implemented, tested and common throughout the roadwayinfrastructure and vehicle fleet. This is analogous to the IP protocolsthat are commonly used at the base implementation of the Internet. Likewith IP, which is controlled and informed through an open process wheredevelopers refer and add to the RFCs (Requests for Comment),¹⁶ therebykeeping all IP related and protocol implementations common andcompatible, a similar process will be required for maintaining anintegrated autonomous vehicle infrastructure system. As indicatedearlier, a common evolving set of design guidelines will be definedeither by a private standard group like SAE or by the Government (aconsortium of States and/or the Department of Transportation).Furthermore, key common message formats for V2V/V2I/V2P messaging,common behaviors (at intersections, when changing lanes, whenmaintaining lane position, when merging, etc.), common error mitigations(emergency stops, or shoulder entry maneuvers, means for notifyingdrivers to take control, means of mitigation when drivers fail to takecontrol after notification, etc.), common driver vehicle driving andstatus interfaces, and common means for indicating to other vehicles anddrivers the operating mode of the vehicles in autonomous mode must beshared across the entire industry so that commonality of the system canbe achieved and maintained. Additionally, as issues, errors, andproblems are identified (which we anticipate would be a continuous andlong-term process), a means for mandatory reporting and publishing ofproblem mitigations should be established and operated as part of thesystem certification process. ¹⁶ RFC: Request For Comment—maintained forthe Internet by Internet Engineering Task Force,https://www.rfc-editor.org/retrieve/

The architecture as disclosed provides a means for implementing mixed AVand manual driving zones within which accident rate can be brought downto nearly zero. We would suggest any lesser goal for autonomous drivingtechnology over public roads is negligent.¹⁷ ¹⁷ Chao, Elain L., Sec.Dept. of Trans., “Automated Driving Systems 2.0: A Vision For Safety”,https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13069a-ads2.0_090617_v9a_tag.pdf

1. A transportation system for managing vehicular behavior, comprising:one or more autonomous vehicles (AVs), each including: (a) a drive trainincluding a source of motive power coupled to wheels, a steering system,and controls for acceleration and braking; (b) a first set of sensorsoperative to provide self-location of the AV; (c) a second set ofsensors operative to monitor an environment surrounding the AV; (d) awireless communications interface; (e) a memory storing road mapsincluding information regarding the location of infrastructure, roadwaysand intersections, and route plans including paths from starting pointsto destinations; (f) an electronic controller in communication with (a),(b), (c), (d) and (e), and wherein the electronic controller isoperative to start, stop, steer, and control the speed of the AV toexecute the route plans; one or more manually operated vehicles (MVs),each including: (g) a drive train including a source of motive powercoupled to wheels, a steering system, and controls for acceleration andbraking; (h) a wireless communications interface; (i) a memory storingroad maps including information regarding the location ofinfrastructure, roadways and intersections; (j) an electronic controllerin communication with (g), (h) and (i), and wherein the electroniccontroller is operative to start, stop, steer, and control the speed ofthe MV in accordance with operator commands; one or more infrastructuredevices, each infrastructure device including a wireless communicationsinterface; wherein both the AVs and MVs receive messages from each otherand from the infrastructure devices enabling the AVs and MVs todetermine the status of traffic conditions and infrastructure in theirvicinity; and wherein the AVs and MVs store, and operate is accordancewith, a common set of rules based upon sensor inputs and the messagesexchanged.
 2. The transportation system of claim 1, wherein the firstset of sensors operative to provide self-location of the AV includes oneor more of the following: GPS, 3D point cloud measurements and matching,inertial measurements, and wheel or speed measurement.
 3. Thetransportation system of claim 1, wherein the second set of sensorsoperative to monitor an environment surrounding the AV include one ormore of the following: LADAR LIDAR, RADAR, and optical, ranging orstereo computer vision.
 4. The transportation system of claim 1,including an AV with an electronic controller operative to performobject recognition or image learning in conjunction with a sensor input.5. The transportation system of claim 1, wherein the route plansexecuted by the AV include a planner operative to generate paths from astarting point to one or more destinations or intermediary points. 6.The transportation system of claim 1, wherein the route plans executedby the AV include generating vehicle controls based on informationassociated with paths that dictate vehicle behavior in proximity tolocations along the paths.
 7. The transportation system of claim 1,wherein the information includes stop signs, speed limits, lane changeprohibitions, or specific tasks executed by AV payloads along the paths.8. The transportation system of claim 1, wherein the road maps stored bythe AVs and MVs include terrain information.
 9. The transportationsystem of claim 1, wherein, in accordance with the rules, the AVs havethe ability to deviate from a planned path due to avoid obstacles orroad conditions.
 10. The transportation system of claim 1, wherein, inaccordance with the rules, the AVs have the ability to perform enhancedbraking and acceleration functions to avoid obstacles and collisions.11. The transportation system of claim 1, further including activelocalization and messaging tagging units carried or worn by pedestriansto inform AVs or MVs as to the position or movement of the pedestrians.12. The transportation system of claim 1, further including activelocalization and messaging tagging units supported on roadway or vehicleprotected objects to inform AVs or MVs as to the position of theobjects.
 13. The transportation system of claim 1, further including AVsthat identify, pick up and place packages, pallets, containers or otherpayloads.
 14. The transportation system of claim 1, further includingAVs with electronic controllers that are programmed to automaticallyperform one or more of the following functions: parking, trailercoupling and detachment, refueling, and vehicle washing or maintenance.15. The transportation system of claim 1, including one or moreinfrastructure devices with sensors operative to detect vehiclesentering or within an intersection and communicate this information toother vehicles entering or within the intersection.
 16. Thetransportation system of claim 1, including one or more infrastructuredevices operative to control intersection lights sequencing ordirectional light functions to improve traffic flow or avoid collisions.